Object identification and tracking systems, such as face recognition and tracking systems, identify areas of a scene captured by a camera that correspond static background features that do not change over time. Identifying these static background features allows such systems to focus image analysis on areas of a captured scene where changes in captured images occur over time. Narrowing the focus of what features in an image should be analyzed as potential objects of interest (e.g., faces) reduces computational complexity associated with image analysis and reduces the time it takes to identify the objects.